Memory controller performing selective and parallel error correction, system including the same and operating method of memory device

ABSTRACT

A memory controller is provided that is configured to control a memory accessed by a device connected to a host processor via a bus. The memory controller is configured to control a memory accessed by a device connected to a host processor via a bus, and includes a first interface circuit configured to communicate with the host processor; a second interface circuit configured to communicate with the memory; an error detection circuit configured to detect an error present in data received from the second interface circuit in response to a first read request received from the first interface circuit; a variable error correction circuit configured to correct the error based on at least one of a reference latency and a reference error correction level included in a first error correction option; and a fixed error correction circuit configured to correct the error in parallel with an operation of the variable error correction circuit.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority under 35 U.S.C. § 119 to Korean PatentApplication No. 10-2020-0154848, filed on Nov. 18, 2020, in the KoreanIntellectual Property Office, the disclosure of which is incorporated byreference herein in its entirety.

FIELD

The inventive concept generally relates to memory devices, and moreparticularly relates to a memory controller that performs selective andparallel error correction, a system including the same, and an operatingmethod of a memory device.

DISCUSSION OF RELATED ART

A system that performs data processing, such as a computing system, mayinclude a central processing unit (CPU), a memory device, input/outputdevices, and a root complex that transmits information between devicesconstituting the system.

The memory device may include a storage class memory (SCM). The SCM hasthe advantage of large capacity and non-volatility, but has longerlatency performance compared to dynamic random-access memory (DRAM).Therefore, if the SCM performs error correction in a data read operationor if latency increases due to garbage collection, or the like, qualityof service (QoS) may deteriorate.

SUMMARY

Embodiments of the present disclosure provide a memory controller thatperforms selective and parallel error correction according to an errorcorrection option set to improve quality of service, a system includingthe same, and an operating method of a memory device.

According to an embodiment of the inventive concept, there is provided amemory controller including a first interface circuit configured tocommunicate with a host processor; a second interface circuit configuredto communicate with a memory; an error detection circuit configured todetect an error present in data read from the second interface circuitin response to a first read request received from the first interfacecircuit; a variable error correction circuit configured to correct thedetected error based on at least one of a reference latency and areference error correction level included in a first error correctionoption; and a fixed error correction circuit configured to correct thedetected error in parallel with an operation of the variable errorcorrection circuit.

According to an embodiment of the inventive concept, there is provided amethod including receiving a first read request from a host processor;transmitting the first read request to a memory and reading datacorresponding to the first read request from the memory; detecting anerror of the read data; correcting the error included in the read databased on at least one of a reference latency or a reference errorcorrection level included in a first error correction option; andproviding first correction data to the host processor.

According to an embodiment of the inventive concept, there is provided asystem including a host processor including at least one core configuredto execute instructions; a memory controller connected to the hostprocessor via a bus; and a memory configured to be accessed through thememory controller, wherein the memory controller may read datacorresponding to the first read request of the host processor from thememory, and perform in parallel a first error correction operation basedon at least one of a reference latency or a reference error correctionlevel included in a first error correction option and a second errorcorrection operation based on a maximum error correction level on anerror included in the data.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the inventive concept will be more clearly understoodfrom the following detailed description taken in conjunction with theaccompanying drawings, in which:

FIG. 1 is a block diagram illustrating a system according to anembodiment of the inventive concept;

FIGS. 2A and 2B are block diagrams illustrating systems according to anembodiment of the inventive concept;

FIG. 3 is a flowchart diagram illustrating a method of setting an errorcorrection option according to an embodiment of the inventive concept;

FIG. 4 is a data structure diagram that illustrates messages accordingto an embodiment of the inventive concept;

FIG. 5 is a block diagram illustrating a system according to anembodiment of the inventive concept;

FIG. 6 is a flowchart illustrating a read operation according to anembodiment of the inventive concept;

FIG. 7 is a flowchart diagram illustrating a selective error correctionoperation according to an embodiment of the inventive concept;

FIG. 8 is a flowchart diagram illustrating a parallel error correctionoperation according to an embodiment of the inventive concept;

FIG. 9 is a data structure diagram that illustrates a response messageaccording to an embodiment of the inventive concept;

FIG. 10 is a block diagram illustrating a system according to anotherembodiment of the inventive concept;

FIGS. 11A and 11B are block diagrams illustrating examples of systemsaccording to an embodiment of the inventive concept; and

FIG. 12 is a block diagram illustrating a data center including a systemaccording to an embodiment of the inventive concept.

DETAILED DESCRIPTION

Hereinafter, various embodiments of the inventive concept will bedescribed with reference to the accompanying drawings.

FIG. 1 illustrates a system 100 according to an embodiment of theinventive concept.

The system 100 may include any computing system or element thereof. Thesystem 100 includes a device 110 and a host processor 120, whichcommunicate with each other. For example, the system 100 may be includedin a stationary computing system such as a desktop computer, a server, akiosk, or the like, or may be included in a portable computing systemsuch as a laptop computer, a mobile phone, a wearable device, or thelike. In addition, in some embodiments, the system 100 may be includedin a system-on-chip (SoC) or a system-in-package (SiP), in which thedevice 110 and the host processor 120 are implemented in one chip orpackage.

As shown in FIG. 1 , the system 100 may include the device 110, the hostprocessor 120, a device-attached memory 130, and a host memory 140. Insome embodiments, the device-attached memory 130 may be omitted from thesystem 100.

Referring to FIG. 1 , the device 110 and the host processor 120 maycommunicate with each other via a link 150 and may perform transmissionor reception of messages therebetween over the link 150. Althoughembodiments of the inventive concept will be described with reference tothe link 150 that is based on the compute express link (CXL)specification supporting CXL protocols, such as under CXL Specification2.0, in alternate embodiments the device 110 and the host processor 120may communicate with each other based on coherent interconnecttechniques, such as, but not limited to, an XBus protocol, an NVLinkprotocol, an Infinity Fabric protocol, a cache coherent interconnect foraccelerators (CCIX) protocol, and/or a coherent accelerator processorinterface (CAPI).

In some embodiments, the link 150 may support multiple protocols, andmessages may be transferred via the multiple protocols. For example, thelink 150 may support CXL protocols including a non-coherent protocol(for example, CXL.io), a coherent protocol (for example, CXL.cache), anda memory access or memory protocol (for example, CXL.mem). In someembodiments, the link 150 may support a protocol, such as, but notlimited to, peripheral element interconnect (PCI), PCI express (PCIe),universal serial bus (USB), or serial advanced technology attachment(SATA). Herein, a protocol supported by the link 150 may also bereferred to as an interconnect protocol.

The device 110 may refer to any device for providing a useful functionto the host processor 120 and, in some embodiments, may correspond to anaccelerator conforming to the CXL specification. For example, softwarerunning on the host processor 120 may offload at least some of itscomputing and/or input/output (I/O) operations to the device 110. Insome embodiments, the device 110 may include at least one of aprogrammable element such as a graphics processing unit (GPU) or aneural processing unit (NPU), a fixed function-providing element such asan intellectual property (IP) core, and a reconfigurable element such asa field programmable gate array (FPGA). As shown in FIG. 1 , the device110 may include a physical layer 111, a multi-protocol multiplexer 112,an interface circuit 113, and an accelerator circuit 114 and maycommunicate with the device-attached memory 130.

The accelerator circuit 114 may perform a useful function, which thedevice 110 provides to the host processor 120, and may also be referredto as accelerator logic. If the device-attached memory 130 is includedin the system 100 as shown in FIG. 1 , the accelerator circuit 114 maycommunicate with the device-attached memory 130, based on a protocolindependent of the link 150 such as a device-specific protocol. Inaddition, as shown in FIG. 1 , the accelerator circuit 114 maycommunicate with the host processor 120 via the interface circuit 113 byusing multiple protocols.

The interface circuit 113 may determine one of the multiple protocols,based on a message for communication between the accelerator circuit 114and the host processor 120. The interface circuit 113 may be connectedto at least one protocol queue included in the multi-protocolmultiplexer 112 and may transmit a message to and receive a message fromthe host processor 120 via the at least one protocol queue. In someembodiments, the interface circuit 113 and the multi-protocolmultiplexer 112 may be integrated into one element. In some embodiments,the multi-protocol multiplexer 112 may include multiple protocol queuesrespectively corresponding to multiple protocols that are supported bythe link 150. In addition, in some embodiments, the multi-protocolmultiplexer 112 may perform arbitration between communications bydifferent protocols and may provide selected communications to thephysical layer 111. In some embodiments, the physical layer 111 may beconnected to a physical layer 121 of the host processor 120 via a singleinterconnect, a bus, a trace, or the like.

The host processor 120 may be a main processor such as a centralprocessing unit (CPU) of the system 100, and, in some embodiments, maycorrespond to a host processor (or a host) conforming to a CXLspecification. As shown in FIG. 1 , the host processor 120 may beconnected to the host memory 140 and may include the physical layer 121,a multi-protocol multiplexer 122, an interface circuit 123, acoherence/cache circuit 124, a bus circuit 125, at least one core 126,and an I/O device 127.

The at least one core 126 may execute instructions and may be connectedto the coherence/cache circuit 124. The coherence/cache circuit 124 mayinclude a cache hierarchy and may also be referred to as coherence/cachelogic. As shown in FIG. 1 , the coherence/cache circuit 124 maycommunicate with the at least one core 126 and the interface circuit123. For example, the coherence/cache circuit 124 may allowcommunication via two or more protocols including a coherent protocoland a memory access protocol. In some embodiments, the coherence/cachecircuit 124 may include a direct memory access (DMA) circuit. The I/Odevice 127 may be used to communicate with the bus circuit 125. Forexample, the bus circuit 125 may be PCIe logic, and the I/O device 127may be a PCIe I/O device.

The interface circuit 123 may allow communication between the device 110and elements, for example, the coherence/cache circuit 124 and the buscircuit 125, of the host processor 120. In some embodiments, theinterface circuit 123 may allow communication of a message and/or databetween the device 110 and the elements of the host processor 120according to multiple protocols, for example, a non-coherent protocol, acoherent protocol, and a memory protocol. For example, the interfacecircuit 123 may determine one of the multiple protocols, based on amessage and/or data for communication between the device 110 and theelements of the host processor 120.

The multi-protocol multiplexer 122 may include at least one protocolqueue. The interface circuit 123 may be connected to the at least oneprotocol queue and may transmit a message to and receive a message fromthe device 110 via the at least one protocol queue. In some embodiments,the interface circuit 123 and the multi-protocol multiplexer 122 may beintegrated into one element. In some embodiments, the multi-protocolmultiplexer 122 may include multiple protocol queues respectivelycorresponding to multiple protocols that are supported by the link 150.In addition, in some embodiments, the multi-protocol multiplexer 122 mayperform arbitration between communications by different protocols andmay provide selected communications to the physical layer 121.

FIGS. 2A and 2B illustrate systems 200 a and 200 b according to anembodiment of the inventive concept.

As shown in FIG. 2A, the system 200 a may include a device 210 a, a hostprocessor 220 a, a device memory 230 a, and a host memory 240 a. In someembodiments, the device memory 230 a may correspond to thedevice-attached memory 130 of FIG. 1 .

Like those described with reference to FIG. 1 , the device 210 a and thehost processor 220 a may communicate with each other based on multipleprotocols. Multiple protocols may include a memory protocol MEM, acoherent protocol COH, and a non-coherent protocol 10. In someembodiments, the memory protocol MEM, the coherent protocol COH, and thenon-coherent protocol 10 may respectively correspond to CXLSpecification 2.0 protocols CXL.mem, CXL.cache, and CXL.io. The memoryprotocol MEM may define transactions between a master and a subordinate.For example, the memory protocol MEM may define transactions from themaster to the subordinate and transactions from the subordinate to themaster. The coherent protocol COH may define interactions between thedevice 210 a and the host processor 220 a. For example, an interface ofthe coherent protocol COH may include three channels including request,response and data. The non-coherent protocol 10 may provide anon-coherent load/store interface for the input/output devices 210 a.

In some embodiments, the device 210 a may correspond to a Type 3 CXLdevice defined in the CXL specification. In this case, the device 210 aand the host processor 220 a may use the memory protocol MEM andnon-coherent protocol 10 to communicate with each other.

The device 210 a may communicate with the device memory 230 a and mayinclude a memory controller 211 a for accessing the device memory 230 a.In some embodiments, different from that shown in FIG. 2A, the memorycontroller 211 a may be outside the device 210 a and may be integratedwith the device memory 230 a. In addition, the host processor 220 a maycommunicate with the host memory 240 a and may include the memorycontroller 221 a for accessing the host memory 240 a. In someembodiments, different from that shown in FIG. 2A, the memory controller221 a may be outside the host processor 220 a and may be integrated withthe host memory 240 a. Referring to FIG. 2B, the system 200 b mayinclude a device 210 b, a host processor 220 b, a device memory 230 b,and a host memory 240 b. The system 200 b of FIG. 2B may be consideredan embodiment of the system 200 a of FIG. 2A. According to an embodimentof the inventive concept, the device 210 b may provide deviceinformation to the host processor 220 b. The host processor 220 b mayprovide an error correction option to the device 210 b to determine anerror correction level and/or an error correction level of the device210 b based on the device information.

The error correction option may be set by the device 210 b itself. Inthis case, the device 210 b may utilize device information and/orinformation of the device memory 230 b. The device 210 b may read datafrom the device memory 230 b according to a read request from the hostprocessor 220 b, and perform an error correction operation according tothe error correction option set with respect to the read data. Thedevice 210 b may provide an error flag indicating error informationtogether with the corrected data to the host processor 220 b. If theabove-described device 210 b is implemented as the Type 3 CXL device,the device 210 b may process the request of the host processor 220 bthrough the memory protocol MEM, and process a device discovery or anerror reporting and management through the non-coherent protocol 10.

According to the present embodiment, the device memory 230 b may beimplemented as various types of memory, and as an example, as a storageclass memory (SCM).

The SCM has the characteristics of non-volatility and a volatile memoryat the same time, and may be accessed in byte units, and thus, its userange is expanding. For example, the SCM may include phase-change RAM(PCM), ferroelectric RAM (FeRAM), magnetic RAM (MRAM), resistive RAM(RRAM), and STT-MRAM.

As one factor for evaluating a quality of service (QoS), a latency, forexample, a tail latency, may be used. In particular, the SCM has theadvantage of large capacity and non-volatility, but has a lower latencyperformance than dynamic random-access memory (DRAM). Therefore, anerror correction operation may be performed in a data read operation, ora latency may increase due to a garbage collection. In addition, becausethe SCM has a non-deterministic latency characteristic, the QoS may bedegraded, and thus it is necessary to reduce the tail latency.

According to a comparative example, various methods may be used toincrease the QoS. As an example, the host processor 220 b may monitor aload level of the device 210 b and control processing of a requesttransmitted to the device 210 b according to the load level.

For example, the device 210 b may determine the load level based onqueued requests, an internal resource usage, and a used bandwidth.Thereafter, the device 210 b may add a field item DevLoad indicating theload level to a response message (e.g., 410 of FIG. 4 ) and transmit theresponse message to the host processor 220. For example, the device 210b may classify the load level into four steps and transmit the loadlevel to the host processor 220 b. If the device 210 b includes aplurality of memories, the device 210 b may classify and transmit theload level with respect to each memory. The host processor 220 b mayperform throttling on the device 210 b based on information includingthe load level provided from the device 210 b.

In addition to the load level, the tail latency may be a factor thatdetermines the QoS. Accordingly, the system 200 b according to thepresent embodiment may perform a series of operations to prevent thetail latency from exceeding a reference value. The host processor 220 bmay stall until a response corresponding to a request provided to thedevice 210 b is received. This may further increase the tail latency.

Accordingly, in order to reduce the tail latency and increase the QoS,the device 210 b according to the present embodiment may first providethe response to the request of the host processor 220 b within a limitnot exceeding a specific latency. For example, as a response to a readrequest, a time taken to transmit corrected data may be predicted toexceed the specific latency. In this case, the device 210 b maysacrifice an accuracy of error correction and provide the host processor220 b with data including an error with a fast latency.

The operation of the device 210 b according to the present embodimentmay be referred to as a selective error correction operation. Theselective error correction operation may mean that while an errorcorrection process does not exceed a reference latency, the errorcorrection operation is performed at a maximum error correction level(or an error free state) of the device 210 b, and while the errorcorrection process is expected to exceed the reference latency, theerror correction operation is performed at a low error correction level.

The host processor 220 b may refer to the message (e.g., 410 in FIG. 4 )transmitted from the device 210 b to determine a maximum value of anerror correction level achievable by the device 210 b (that is, amaximum error correction level) as well as minimum and maximum values ofa latency achievable by the device 210 b.

The maximum error correction level and/or the minimum and maximum valuesof the latency of the device 210 b may be determined based oncharacteristics of the device 210 b such as the performance of thedevice 210 b, the load level, and a type of memory. The error correctionlevel means the accuracy of error correction that the device 210 btargets, and the higher the value is, the more detected errors thedevice 210 b may be corrected.

The host processor 220 b may set an error correction option includingthe reference error correction level and the reference latency of thedevice 210 b. For example, the host processor 220 b may set the errorcorrection option so that the device 210 b performs error correctionhaving a certain accuracy or higher and the error correction operationdoes not exceed the reference latency. An error correction optionsetting message may be the same as 420 of FIG. 4 .

Thereafter, if the host processor 220 b transmits a read request to thedevice 210 b, the device 210 b may perform the error correctionoperation based on the error correction option and respond to the hostprocessor 220 b. The device 210 b may transmit a response message andthe corrected data to the host processor 220 b together within thereference latency. The response message may include an error flagindicating whether there is an error in the corrected data. The hostprocessor 220 b may determine an accuracy of the received corrected databased on the error flag. The host processor 220 b may request data fromwhich the error is completely corrected again from the device 210 b ifnecessary, and the device 210 b may perform the error correctionoperation until the request is received and provide the data from whichthe error is completely corrected to the host processor 220 b.

As described above, the device 210 b performs the selective errorcorrection operation based on the reference latency and the maximumerror correction level, and thus, a time during which the operation ofthe host processor 220 b stalls may be reduced, thereby improving theQoS.

FIG. 3 illustrates a method of setting an error correction optionaccording to an embodiment of the inventive concept.

A host 310 may include the host processor (e.g., 120 in FIG. 1, 220 a inFIG. 2A or 220 b in FIG. 2B) and the host memory (e.g., 140 in FIG. 1,240 a in FIG. 2A or 240 b in FIG. 2B). Further, the memory controller320 may correspond to the memory controller 211 a of FIG. 2A or 211 b ofFIG. 2B.

Referring to FIGS. 2B and 3 together, the host 310 may request deviceinformation from the memory controller 320 (S110).

The memory controller 320 may transmit information about the device 210b to the host 310 (S120). Operation S120 may be performed bytransmitting a response message to operation S110. For example,operation S120 may be performed through a Subordinate to Master (S2M)transaction of the memory protocol CXL.mem, and as an example, theresponse message may be as 410 of FIG. 4 .

The host 310 may set the error correction option of the device 210 bbased on information included in the response message (S130). The errorcorrection option may be set through an M2S transaction of the memoryprotocol CXL.mem. The host 310 may set the error correction option,which is a reference for an error correction operation of the device 210b, based on the device information received in operation S120. An errorcorrection option setting message may include an error correction leveland/or a reference latency based on whether the device 210 b performsthe error correction operation, as shown in 420 of FIG. 4 . As anotherexample, the memory device 320 may set the error correction option inconsideration of the error correction option received from the host 310and device information or information of the device memory 230 aattached to the device.

The above-described operations S110 to S130 may be performed bytransmitting any request from the host 310 to the memory controller 320.As another example, operations S110 to S130 may be performed during abooting stage of a system.

FIG. 4 illustrates messages according to an embodiment of the inventiveconcept.

Referring to FIGS. 2A, 2B, 3 and 4 together, a response message 410 maybe a message transmitted in operation S120 of FIG. 3 , and an errorcorrection option setting message 420 may be a message transmitted inoperation S130 of FIG. 3 .

The response message 410 may include first to fourth fields F1 to F4,and may further include additional fields. Each of the fields F1 to F4included in the response message 410 may include at least one bit, andinformation corresponding to a value of the at least one bit may bedefined by a protocol.

For example, the first field F1 is a valid signal and may include 1 bitindicating that the response message 410 is a valid message. The secondfield F2 is an operation code and may include a plurality of bitsdefining an operation corresponding to the response message 410. Forexample, the operation code of the second field F2 may indicateinvalidation for updating metadata. The third field F3 is an identifierof a logical device (LD), and may be included if the device 210 b isimplemented as a multi-logical device (MLD). The fourth field F4 is aload level, and may include a plurality of bits indicating a load levelof the device 210 b at the current time.

The response message 410 may include device information including aminimum value of latency achievable by the device 210 b, and may furtherinclude various types of information required to perform a selectiveerror correction operation.

The error correction option setting message 420 may include fifth toeighth fields F5 to F8, and may further include additional fields. Eachof the fields F5 to F8 included in the error correction option settingmessage 420 may include at least one bit, and information correspondingto a value of the at least one bit may be defined by a protocol.

The fifth field F5 and the sixth field F6 of the error correction optionsetting message 420 may be similar to the first field F1 and the secondfield F2 of the response message 410, respectively, and thus redundantdescriptions thereof may be omitted.

The seventh field F7 is a reference error correction level, and mayindicate a target accuracy when the device 210 b performs the errorcorrection operation. The device 210 b may perform the error correctionoperation based on the reference error correction level included in theseventh field F7.

The eighth field F8 is latency information, and may include a referencelatency for determining whether the device 210 b performs the selectiveerror correction operation. The reference latency may be a value equalto or greater than the minimum latency achievable by the device 210 b.According to the error correction option setting method, the referencelatency may be a value less than or equal to the maximum latencyachievable by the device 210 b.

For example, the device 210 b may perform the error correction operationaiming at an accuracy equal to or higher than an error correction levelwithin a limit not exceeding the reference latency.

If the device 210 b is implemented as an MLD, the host processor 220 bmay set different error correction options with respect to each of thelogical devices 210 b. In this case, the error correction option settingmessage 420 may further include a logical device identifier field LD-ID.

FIG. 5 illustrates a system 500 according to an embodiment of theinventive concept.

Referring to FIG. 5 , a memory controller 520 may include a firstinterface circuit 521, a second interface circuit 522, an errorcorrection code (ECC) setting circuit 523, a variable ECC circuit 524, afixed ECC circuit 525, a buffer 526 and an error detector 527. Thememory controller 520 may correspond to one of the above-describedmemory controllers (e.g., 211 a and 211 b in FIGS. 2A and 2B, and 320 inFIG. 3 ). The host 510 may include a host processor and a host memory ofFIGS. 1, 2A, and/or 2B.

The first interface circuit 521 may communicate with the host 510. In anembodiment, the first interface circuit 521 may communicate with thehost 510 through an accelerator circuit (e.g., 114 in FIG. 1 ). Thefirst interface circuit 521 may receive an error correction optionsetting message Msg_ECC from the host 510 and may receive a read orwrite request REQ. Also, the first interface circuit 521 may transmit atleast one of the read data RD, first corrected data FD1 and secondcorrected data FD2 to the host 510 together with a response to the readrequest.

The second interface circuit 522 may communicate with the memory 530.The memory 530 may refer to any storage medium capable of storinginformation. For example, the memory 530 may be manufactured by asemiconductor process, may include a volatile memory such as staticrandom-access memory (SRAM), dynamic random-access memory (DRAM), etc.,and may also include a non-volatile memory such as flash memory,resistive random-access memory (RRAM), etc. As described above, thememory 530 may include an SCM. Also, the memory 530 may include aplurality of logic devices. The second interface circuit 522 may providea command and an address to the memory 530 and may receive a read dataRD from the memory 530.

The ECC setting circuit 523 may receive the error correction optionsetting message Msg_ECC from the host 510 through the first interfacecircuit 521 and control operations of the error detector 527 and/or thevariable ECC circuit 524 according to each field value of the errorcorrection option setting message Msg_ECC. The ECC setting circuit 523may provide an error correction level and/or a reference latency thatare references for performing a selective error correction operation asa plurality of components.

The error detector 527 may detect whether there is an error in the readdata RD read from the memory 530 and determine whether to correct theerror. The error detector 527 may provide the read data with error RD_Eto the variable ECC circuit 524 and/or the fixed ECC circuit 525 forerror correction. If it is determined that read data RD is error-free,the error detector 527 may store the read data RD in the buffer 526.

As an example, the error detector 527 may determine whether to performerror correction of the read data RD with an accuracy of the referenceerror correction level within the reference latency based on the errorcorrection level and/or the reference latency provided from the ECCsetting circuit 523. If it is determined that error correction of theread data is performed, the error detector 527 may transmit the readdata with error RD_E to the fixed ECC circuit 525, and if it isdetermined that error correction of the read data is not performed, theerror detector 527 may transmit the read data with error RD_E to each ofthe variable ECC circuit 524 and the fixed ECC circuit 525.

In addition, the error detector 527 may include an error flag in aresponse message to a read request from the host 510. The error flag isa field indicating whether the error is included in the read data, andmay have a value of 0 or 1.

The variable ECC circuit 524 and the fixed ECC circuit 525 may perform aseries of operations for performing error correction of the read datawith error RD_E read from the memory 530. For example, the variable ECCcircuit 524 and the fixed ECC circuit 525 may correct an error bitincluded in the read data with error RD_E through an ECC decodingoperation.

The variable ECC circuit 524 may perform error correction of the readdata with error RD_E based on the reference error correction levelprovided from the ECC setting circuit 523. The variable ECC circuit 524may provide the first corrected data FD1 to the host 510 through thefirst interface circuit 521.

The fixed ECC circuit 525 may not receive the reference error correctionlevel from the ECC setting circuit 523 and may perform error correctionof the read data with error RD_E based on a maximum accuracy. Forexample, the fixed ECC circuit 525 may generate the second correcteddata FD2 in an error-free state. The fixed ECC circuit 525 may store thesecond corrected data FD2 in the buffer 526.

The memory controller 520 may repeatedly receive a read request for thesame data from the host 510. As an example, the memory controller 520may determine whether there is a redundant request for the same databased on an address included in the read request from the host 510. Aread request received first according to a time order may be referred toas a first read request, and a read request received later may bereferred to as a second read request.

The read data may be stored in the buffer 526 as a result of performingan operation for the first read request. In this regard, the memorycontroller 520 may fetch data stored in the buffer 526 as a response tothe second read request and transmit the data to the host 510. At thistime, the second corrected data FD2 in an error-free state may betransmitted to the host 510. An operation of generating a read command,providing the read command to the memory 530, and receiving data readfrom the memory 530 may be bypassed, and thus latency may be improved.

FIG. 6 illustrates a read operation according to an embodiment of theinventive concept.

Referring to FIG. 6 , a device 620 according to the present embodimentmay be a Type 3 CXL device defined in the CXL specification. A host 610may include a host processor and a host memory of FIGS. 1 to 2 . In thisregard, the host 610 may regard the device 620 as a disaggregatedmemory. According to the above-described memory protocol MEM, a mastermay correspond to the host 610 and a subordinate may correspond to thedevice 620. In addition, each operation described later may be performedthrough an M2S transaction or an S2M transaction of the memory protocolCXL.mem.

The host 610 may transmit a read request to a memory controller 621(S210). As an example, the host 610 may set an error correction optionof the device 620 through an operation of FIG. 3 before transmitting theread request. As another example, the host 610 may transmit an errorcorrection option setting message of the device 620 together with theread request. As another example, the error correction option of thedevice 620 may be set by the device 620 itself. The information requiredto set the error correction option may include the device informationdescribed with reference to FIG. 3 , and may further include variousother information.

The memory controller 621 may transmit the read request to the memory622 (S220) and receive the read data (S230).

The memory controller 621 may determine whether there is an error byperforming an error detection operation on the read data (S240).

If there is an error in the read data, the memory controller 621 mayperform the error correction operation of the read data according to theset error correction option (S250). For example, the memory controller621 may determine whether the error correction operation may beperformed on the detected error at a reference error correction levelwithin a reference latency. Based on a result of the determination, thememory controller 621 may make the variable ECC circuit and/or the fixedECC circuit perform the error correction operation.

Thereafter, the memory controller 621 may transmit the corrected data tothe host 610 (S260). The corrected data may be in an error-free state ora state including the error according to the reference error correctionlevel. As an example, if the error correction option is set by assigninga maximum weight to a latency, the memory controller 621 may immediatelytransmit the read data to the host 610 without correcting the error.

In this regard, an error flag may be used to indicate whether the dataincludes the error. For example, as the response to the read request inoperation S210, the memory controller 621 may generate a responsemessage including an error flag (e.g., 700 in FIG. 10 ) and transmit theresponse message to the host 610 together with the data. The error flagmay indicate whether the corrected data to be transmitted includes theerror, and may have a value of 0 or 1.

If there is no error in the data read in operation S240, the memorycontroller 621 may transmit the read data to the host 610 (S260). Inthis case, the error flag of the response message may have a differentvalue from the error flag if there is an error.

FIG. 7 illustrates a selective error correction operation according toan embodiment of the inventive concept.

FIG. 7 may illustrate an embodiment of operations S250 and S260 of FIG.6 , without limitation. Hereinafter, a process will be described afterdata is read from the memory 622 according to a read request from thehost 610.

Referring to FIGS. 6 and 7 together, the memory controller 621 maydetect an error in the data read from the memory 622 (S310). This maycorrespond to operation S240 of FIG. 6 .

The memory controller 621 may determine whether the detected error iscorrectable within a reference latency (S320). The memory controller 621may additionally consider an error correction level. For example, thememory controller 621 may determine whether error correction is possibleat a host-selected reference error correction level within thehost-selected reference latency limit.

If it is determined that the detected error is correctable in operationS320, the memory controller 621 may correct the detected error through afixed ECC circuit (S330). The fixed ECC circuit may correct the readdata in an error-free state regardless of the set error correctionlevel. The corrected data may be stored in a buffer through the fixedECC circuit.

The memory controller 621 may transmit the corrected data to the host(S340). The memory controller 621 may transmit a response message alongwith the corrected data, and the response message may include contentindicating that the corrected data is in the error-free state.

If it is determined that the detected error is not correctable at thereference error correction level within the reference latency inoperation S320, the memory controller 621 may perform an operation A andcorrect the detected error through a variable ECC circuit in parallel(S350). Operation A may be performed in parallel with various othertasks of the device 620, and descriptions thereof are given withreference to FIG. 8 . Because the variable ECC circuit corrects theerror based on the reference error correction level, the error may beincluded in the corrected data.

The memory controller 621 may transmit the response message includingthe corrected data and an error flag to the host (S360). The correcteddata may not be in the error-free state by the variable ECC circuit.Therefore, the error flag of the response message may indicate 1. Thecorrected data may be transmitted to the host 610 by the variable ECCcircuit without passing through a buffer.

According to the above-described embodiments, the host 610 may receivedata within the reference latency. In addition, the host 610 mayselectively request data according to a situation, thereby managingresources efficiently.

FIG. 8 illustrates a parallel error correction operation according to anembodiment of the inventive concept.

As described above, if the memory controller 621 determines that anerror is not correctable at a reference error correction level within areference latency according to an error correction option, the memorycontroller 621 may perform an error correction operation through avariable ECC circuit. In this regard, the memory controller 621 mayprovide read data to a fixed ECC circuit in parallel with an operationof the variable ECC circuit. Accordingly, the variable ECC circuit andthe fixed ECC circuit may perform the error operation in parallel sothat first correction data corrected to a reference error correctionlevel and second correction data corrected to an error-free state may begenerated.

The operation of the variable ECC circuit is limited to a set rate, andthus, the first correction data may be generated at a faster rate thanthe second correction data. Accordingly, the first correction data maybe provided to the host 610 within the latency set by the host 610.After receiving the first correction data, the host 610 may request thedata in the error-free state again. In this case, the memory controller621 may bypass a process of reading the data from the memory 622 anddetecting the error (e.g., S220 to S240 in FIG. 6 ) and immediatelytransmit the second correction data to the host 610. As a result, thedevice 620 may improve latency with respect to a read operation andprovide data with high accuracy to the host 610.

Referring to FIGS. 6 and 8 , the memory controller 621 may perform theerror correction operation through the fixed ECC circuit in parallelwith operation S350 of FIG. 7 to store the corrected data in a buffer(S410). That is, the memory controller 621 may transmit the read data tothe host 610 in response to a read request, and perform error correctionof the read data separately. Thus, the data in the error-free state maybe stored in the buffer.

Thereafter, the memory controller 621 may re-receive the read requestfor the same data from the host 610 (S420). The read request for thesame data may be performed as a read retry or a delayed read, withoutlimitation. The memory controller 621 may determine whether there is arepeat request for the same data based on an address included in theread request.

The memory controller 621 may determine a cache hit (S430). For example,the memory controller 621 may confirm whether data corresponding to theaddress included in the read request from the host 610 is stored in thebuffer and determine there is a cache hit.

The memory controller 621 may transmit the corrected data stored in thebuffer to the host 610 (S440). The memory controller 621 may transmit aresponse message to the read request received in operation S420 togetherwith the corrected data to the host 610. In this regard, because thecorrected data is in the error-free state, an error flag of anerror-free response message may indicate 0.

FIG. 9 illustrates a response message 700 according to an embodiment ofthe inventive concept.

Referring to FIGS. 5 and 9 together, the response message 700 may begenerated based on an S2M Data Responses (DRS) message of the CXLprotocol. The response message 700 may include a plurality of fields,and descriptions redundant with those given above with reference to FIG.4 are omitted.

The response message 700 according to the present embodiment may begenerated as a response to a read request from the host 510 and mayinclude an error flag (reserved) in a ninth field F9. The error flag mayindicate whether an error is included in read data transmitted to thehost 510 together with the response message 700.

For example, the error may be corrected by the variable ECC circuit 524and/or the fixed ECC circuit 525 according to a selective errorcorrection operation. For example, if the error is corrected by thevariable ECC circuit 524, the error flag may be set to 1 when there isan error in the corrected data. As another example, if the fixed ECCcircuit 525 corrects data in an error-free state, the error flag may beset to 0.

The host 510 may receive the response message 700 and perform an errorcorrection operation on its own or request data from the memorycontroller 520 again based on the error flag included in the responsemessage 700.

FIG. 10 illustrates a system 800 according to another embodiment of theinventive concept. A case in which the above-described embodiments ofthe inventive concept are applied to a system including a type 3 CXLdevice defined in the CXL specification is described with reference toFIG. 10 , but is not limited thereto, and the above-describedembodiments may also be applied to a type 1 device or a type 2 devicedefined in the CXL specification.

Referring to FIG. 10 , the system 800 may include a root complex 810, aCXL memory expander 820 connected thereto, and a memory 830. The rootcomplex 810 may correspond to a host processor (e.g., 120 of FIG. 1, 220a of FIG. 2A, 220 b of FIG. 2B). The root complex 810 may communicatewith a host memory through a memory controller, the memory controllermay correspond to the memory controller 221 a of FIG. 2B, and the hostmemory may correspond to the host memory 240 a of FIG. 2B. The rootcomplex 810 may include a home agent and an input/output bridge. Thehome agent may communicate with the CXL memory expander 820 based on thememory protocol CXL.mem. Based on the CXL protocol, the home agent maycorrespond to an agent at the host side disposed to resolve the overallcoherence of the system 800 with respect to a given address. Theinput/output bridge may communicate with the CXL memory expander 820based on the non-coherent protocol CXL.io and include the input/outputmemory management unit IOMMU. The CXL memory expander 820 corresponds toa device (for example, 110 of FIG. 1, 210 a of FIG. 2A, 210 b of FIG.2B), thereby performing a selective and/or parallel error correctionoperation according to the above-described embodiments.

The CXL memory expander 820 may include a memory controller 821. Thememory controller 821 may perform operations of the memory controllers(211 of FIG. 2, 320 of FIG. 3, 520 of FIG. 5, and 621 of FIG. 6 )described above with reference to FIGS. 1 to 9 .

The memory controller 821 may be referred to as a memory interfacecircuit, and may access the memory 830 based on an interface of thememory 830. Further, according to an embodiment of the inventiveconcept, the CXL memory expander 820 may output data to the root complex810 through the input/output bridge based on the non-coherent protocolCXL.io or a similar PCIe. The memory 830 may refer to any medium forstoring data. As an example, the memory 830 may include a semiconductormemory device, for example, a non-volatile memory device such as a flashmemory, a resistive random-access memory (RRAM), etc. and/or a volatilememory device such as a dynamic random-access (DRAM) memory), a staticrandom-access memory (SRAM), etc.

The memory 830 may include a plurality of memory areas M1 to Mn, andeach of the memory areas M1 to Mn may be implemented as various units ofmemory. As an example, if the memory 830 includes a plurality ofvolatile or non-volatile memory chips, a unit of each of the memoryareas M1 to Mn may be a memory chip. Alternatively, the memory 830 maybe implemented so that the unit of each of the memory areas M1 to Mncorresponds to various sizes defined in the memory, such as asemiconductor die, a block, a bank, and a rank.

According to an embodiment, the plurality of memory areas M1 to Mn mayhave a hierarchical structure. For example, the first memory area M1 maybe an upper level memory, and the n-th memory area Mn may be a lowerlevel memory. The higher level memory may have a relatively smallcapacity and a faster response speed, and the lower level memory mayhave a relatively large capacity and a slower response speed. Due tothis difference, an achievable minimum latency (or maximum latency) or amaximum error correction level of each memory area may be different.

Accordingly, a host may set an error correction option for each of thememory areas M1 to Mn. In this case, the host may transmit a pluralityof error correction option setting messages to the memory controller821. Each error correction option setting message may include areference latency, a reference error correction level, and an identifierfor identifying a memory area. Accordingly, the memory controller 821may confirm a memory area identifier of the error correction optionsetting message and set the error correction option for each of thememory areas M1 to Mn.

As another example, a variable ECC circuit or a fixed ECC circuit mayperform an error correction operation according to a memory area inwhich data to be read is stored. For example, data of higher importancemay be stored in a higher level memory, and accuracy may be weightedrather than latency. Accordingly, the operation of the variable ECCcircuit may be omitted for data stored in the upper level memory, andthe error correction operation may be performed by the fixed ECCcircuit. As another example, data of low importance may be stored in thelower level memory. The data stored in the lower level memory isweighted to the latency, and thus, the operation by the fixed ECCcircuit may be omitted. That is, the read data may be immediatelytransmitted to the host without error correction or the error correctionoperation by the variable ECC circuit in response to the read request.According to the importance of data and the memory area in which thedata is stored, the selective and parallel error correction operationsmay be performed in various ways and are not limited to theabove-described embodiment.

The memory area identifier may also be included in a response message ofthe memory controller 821. A read request message may include the memoryarea identifier together with an address of the data to be read. Theresponse message may include the memory area identifier with respect tothe memory area including the read data.

FIGS. 11A and 11B illustrate examples of systems 900 a and 900 baccording to an embodiment of the inventive concept.

Specifically, the block diagrams of FIGS. 11A and 11B illustrate thesystems 900 a and 900 b including multiple CPUs. In the followingdescriptions regarding FIGS. 11A and 11B, repeated descriptions areomitted.

Referring to FIG. 11A, the system 900 a may include first and secondCPUs 11 a and 21 a and may include first and second double data rate(DDR) memories 12 a and 22 a, which are respectively connected to thefirst and second CPUs 11 a and 21 a. The first and second CPUs 11 a and21 a may be connected to each other via an interconnection system 30 athat is based on a processor interconnection technique. As shown in FIG.11A, the interconnection system 30 a may provide at least one CPU-to-CPUcoherent link.

The system 900 a may include a first I/O device 13 a and a firstaccelerator 14 a, which communicate with the first CPU 11 a, and mayinclude a first device memory 15 a connected to the first accelerator 14a. The first CPU 11 a may communicate with the first I/O device 13 a viaa bus 16 a and may communicate with the first accelerator 14 a via a bus17 a. In addition, the system 900 a may include a second I/O device 23 aand a second accelerator 24 a, which communicate with the second CPU 21a, and may include a second device memory 25 a connected to the secondaccelerator 24 a. The second CPU 21 a may communicate with the secondI/O device 23 a via a bus 26 a and may communicate with the secondaccelerator 24 a via a bus 27 a.

Protocol-based communication may be performed through the buses 16 a, 17a, 26 a, and 27 a, and a protocol may support the selective and parallelerror correction operations described above with reference to thedrawings. Accordingly, the latency required for the error correctionoperation with respect to a memory, for example, the first device memory15 a, the second device memory 25 a, the first DDR memory 12 a and/orthe second DDR memory 22 a may be reduced and the performance of thesystem 900 a may be improved.

Referring to FIG. 11B, similar to the system 900 a of FIG. 11A, thesystem 900 b may include first and second CPUs 11 b and 21 b, first andsecond DDR memories 12 b and 22 b, first and second I/O devices 13 b and23 b, and first and second accelerators 14 b and 24 b, and may furtherinclude a remote far memory 40. The first and second CPUs 11 b and 21 bmay communicate with each other via an interconnection system 30 b. Thefirst CPU 11 b may be connected to the first and second I/O devices 13 band 23 b via buses 16 b and 17 b, respectively, and the second CPU 21 bmay be connected to the first and second accelerators 14 b and 24 b viabuses 26 b and 27 b, respectively.

The first and second CPUs 11 b and 21 b may be connected to the remotefar memory 40 via first and second buses 18 and 28, respectively. Theremote far memory 40 may be used for the extension of memory in thesystem 900 b, and the first and second buses 18 and 28 may be used asmemory extension ports. A protocol corresponding to the first and secondbuses 18 and 28 as well as the buses 16 a, 17 a, 26 a, and 27 a may alsosupport the selective and parallel error correction operations describedabove with reference to the drawings. Accordingly, the latency requiredfor error correction with respect to the remote far memory 40 may bereduced and the performance of the system 900 b may be improved.

FIG. 12 illustrates a data center 1 including a system according to anembodiment of the inventive concept.

In some embodiments, the system described above with reference to thedrawings may be included in the data center 1 as an application serverand/or a storage server. In addition, an embodiment related to selectiveand parallel error correction operations of a memory controller appliedto the embodiments of the inventive concept may be applied to each ofthe application server and the storage server.

Referring to FIG. 12 , the data center 1 may collect various data andprovide services and may be referred to as a data storage center. Thedata center 1 may be a system for operating a search engine and adatabase and may be a computing system used in companies, such as banks,or in government agencies. As shown in FIG. 12 , the data center 1 mayinclude application servers 50_1 to 50_n and storage servers 60_1 to60_m (m and n are integers greater than 1). The number n of applicationservers 50_1 to 50_n and the number m of storage servers 60_1 to 60_mmay be variously selected according to embodiments, and the number n ofapplication servers 50_1 to 50_n may be different from the number m ofstorage servers 60_1 to 60_m.

The application servers 50_1 to 50_n may include at least one ofprocessors 51_1 to 51_n, memories 52_1 to 52_n, switches 53_1 to 53_n,network interface controllers (NICs) 54_1 to 54_n, and storage devices55_1 to 55_n The processors 51_1 to 51_n may control all operations ofthe application servers 50_1 to 50_n and may access the memories 52_1 to52_n to execute instructions and/or data loaded into the memories 52_1to 52_n. The memories 52_1 to 52_n may include, but are not limited to,double data-rate synchronous DRAM (DDR SDRAM), high bandwidth memory(HBM), a hybrid memory cube (HMC), a dual in-line memory module (DIMM),an Optane DIMM, or a non-volatile DIMM (NVMDIMM).

According to embodiments, the respective numbers of processors andmemories, which are included in the application servers 50_1 to 50_n,may be variously selected. In some embodiments, the processors 51_1 to51_n and the memories 52_1 to 52_n may provide processor-memory pairs.In some embodiments, the number of processors 51_1 to 51_n may bedifferent from the number of memories 52_1 to 52_n. The processors 51_1to 51_n may include single-core processors or multi-core processors. Insome embodiments, as illustrated by a dashed line in FIG. 12 , thestorage devices 55_1 to 55_n may be omitted from the application servers50_1 to 50_n. The number of storage devices 55_1 to 55_n included in theapplication servers 50_1 to 50_n may be variously selected according toembodiments. The processors 51_1 to 51_n, the memories 52_1 to 52_n, theswitches 53_1 to 53_n, the NICs 54_1 to 54_n, and/or the storage devices55_1 to 55_n communicate with each other through the link describedabove with reference to the drawings.

The storage servers 60_1 to 60_m may include at least one of processors61_1 to 61_m, memories 62_1 to 62_m, switches 63_1 to 63_m, NICs 64_1 to64_n, and storage devices 65_1 to 65_m. The processors 61_1 to 61_m andthe memories 62_1 to 62_m may operate similarly to the processors 51_1to 51_n and the memories 52_1 to 52_n of the application servers 50_1 to50_n described above.

The application servers 50_1 to 50_n and the storage servers 60_1 to60_m may communicate with each other via a network 70. In someembodiments, the network 70 may be implemented by using Fibre Channel(FC), Ethernet, etc. The FC may be a medium used for relativelyhigh-speed data transmission and may use an optical switch providinghigh performance and/or high availability. According to access methodsof the network 70, the storage servers 60_1 to 60_m may be provided asfile storage, block storage, or object storage.

In some embodiments, the network 70 may include a storage-dedicatednetwork such as a storage area network (SAN). For example, the SAN mayuse an FC network and may be an FC-SAN implemented according to the FCProtocol (FCP). Alternatively, the SAN may include an Internet Protocol(IP)-SAN, which uses a Transmission Control Protocol (TCP)/IP networkand is implemented according to an Internet Small Computer SystemInterface (iSCSI, or SCSI over TCP/IP) protocol. In some embodiments,the network 70 may include a general network such as a TCP/IP network.For example, the network 70 may be implemented according to a protocolsuch as FC over Ethernet (FCoE), Network Attached Storage (NAS), or NVMeover Fabrics (NVMe-oF).

Hereinafter, the application server 50_1 and the storage server 60_1will be mainly described, but descriptions of the application server50_1 may also be applied to another application server (e.g., 50_n), anddescriptions of the storage server 60_1 may also be applied to anotherstorage server (e.g., 60_m).

The application server 50_1 may store data, which a user or a client hasrequested to store, in one of the storage servers 60_1 to 60_1 m via thenetwork 70. In addition, the application server 50_1 may obtain data,which a user or a client has requested to read, from one of the storageservers 60_1 to 60_1 m via the network 70. For example, the applicationserver 50_1 may be implemented by a web server, a database managementsystem (DBMS), or the like.

The application server 50_1 may access a memory 52_n or a storage device55_n, which is included in the other application server 50_n, via thenetwork 70 and/or may access the memories 62_1 to 62_m or the storagedevices 65_1 to 65_m respectively included in the storage servers 60_1to 60_m via the network 70. Thus, the application server 50_1 mayperform various operations on data stored in the application servers50_1 to 50_n and/or the storage servers 60_1 to 60_m. For example, theapplication server 50_1 may execute instructions for moving or copyingdata between the application servers 50_1 to 50_n and/or the storageservers 60_1 to 60_m. Here, the data may be moved from the storagedevices 65_1 to 65_m of the storage servers 60_1 to 60_m to the memories52_1 to 52_m of the application servers 50_1 to 50_n, directly orthrough the memories 62_1 to 62_m of the storage servers 60_1 to 60_m.In some embodiments, the data moved via the network 70 may be dataencrypted for security or privacy.

In the storage server 60_1, an interface IF may provide physicalconnection between the processor 61_1 and a controller CTRL and physicalconnection between the NIC 64_1 and the controller CTRL. For example,the interface IF may be implemented in a direct-attached storage (DAS)manner, in which a connection to the storage device 65_1 is directlymade by a dedicated cable. In addition, for example, the interface IFmay be implemented in various interface manners such as AdvancedTechnology Attachment (ATA), Serial ATA (SATA), external SATA (e-SATA),Small Computer Small Interface (SCSI), Serial Attached SCSI (SAS),Peripheral Component Interconnect (PCI), PCI express (PCIe), NVM express(NVMe), IEEE 1394, universal serial bus (USB), a secure digital (SD)card, a multi-media card (MMC), an embedded multi-media card (eMMC),Universal Flash Storage (UFS), embedded Universal Flash Storage (eUFS),and a compact flash (CF) card interface, or the like.

In the storage server 60_1, the switch 63_1 may selectively connect theprocessor 61_1 to the storage device 65_1 or selectively connect the NIC64_1 to the storage device 65_1, according to control by the processor61_1.

In some embodiments, the NIC 64_1 may include a network interface card,a network adaptor, etc. The NIC 64_1 may be connected to the network 70by a wired interface, a wireless interface, a Bluetooth interface, anoptical interface, etc. The NIC 64_1 may include an internal memory, adigital signal processor (DSP), a host bus interface, etc. and may beconnected to the processor 61_1 and/or the switch 63_1. In someembodiments, the NIC 64_1 may be integrated with at least one of theprocessor 61_1, the switch 63_1, and the storage device 65_1.

In the application servers 50_1 to 50_n or the storage servers 60_1 to60_m, the processors 51_1 to 51_m and 61_1 to 61_n may transmit commandsto the storage devices 55_1 to 55_n and 65_1 to 65_m or the memories52_1 to 52_n and 62_1 to 62_m and thus program data thereto or read datatherefrom. Here, the data may be data that is error-corrected by anerror correction code (ECC) engine. The data may be data havingundergone data bus inversion (DBI) or data masking (DM) and may includeCRC information. The data may be data encrypted for security or privacy.

The storage devices 55_1 to 55_n and 65_1 to 65_m may transmit controlsignals and command/address signals to a non-volatile memory device NVM(e.g., a NAND flash memory device) in response to read commands receivedfrom the processors 51_1 to 51_m and 61_1 to 61_n Accordingly, when datais read from the non-volatile memory device NVM, a read enable signalmay be input as a data output control signal and thus function to causethe data to be output to a DQ bus. A data strobe may be generated byusing the read enable signal. A command and an address signal may belatched according to rising edges or falling edges of a write enablesignal.

The controller CTRL may take overall control of operations of thestorage device 65_1. In an embodiment, the controller CTRL may includestatic random-access memory (SRAM). The controller CTRL may write datato the non-volatile memory device NVM in response to a write command ormay read data from the non-volatile memory device NVM in response to theread command. For example, the write command and/or the read command maybe generated based on a request provided by a host, for example, theprocessor 61_1 in the storage server 60_1, the processor 61_m in theother storage server 60_m, or the processors 51_1 to 51_n in theapplication servers 50_1 and 50_n. A buffer BUF may temporarily store(buffer) data which is to be written to or has been read from thenon-volatile memory device NVM. In addition, the buffer BUF may storeDRAM. Also, the buffer BUF may store meta data, and the meta data mayrefer to user data or data generated by the controller CTRL to managethe non-volatile memory device NVM. The storage device 65_1 may includea secure element (SE) for security or privacy.

While the inventive concept has been particularly shown and describedwith reference to embodiments thereof, it will be understood by those ofordinary skill in the pertinent art that various changes in form anddetails may be made therein without departing from the spirit and scopeof the present disclosure as set forth in the following claims.

1-21. (canceled)
 22. A method performed by a Compute Express Link™ (CXL)device connected to a host processor, the method comprising: receiving afirst read request from the host processor based on a CXL memoryprotocol; transmitting the first read request to a memory and readingfirst read data corresponding to the first read request from the memory;detecting an error of the first read data; correcting the error includedin the first read data based on at least one of a reference latency or areference error correction level included in a first error correctionoption; and providing first correction data to the host processor basedon the CXL memory protocol.
 23. The method of claim 22, furthercomprising: receiving a device information request from the hostprocessor based on a CXL input/output protocol; transmitting deviceinformation to the host processor; and receiving the first errorcorrection option from the host processor.
 24. The method of claim 23,wherein the device information is based on a subordinate-to-master (S2M)transaction, wherein the device information comprises: a first fielddata corresponding to a logical device identifier, and a second fielddata corresponding to a load level, wherein the load level is determinedbased on at least one of a queued request, an internal resource usage ora bandwidth usage.
 25. The method of claim 23, wherein the first errorcorrection option is based on a master-to-subordinate (M2S) transaction.26. The method of claim 22, wherein the correcting of the error includedin the first read data comprises, according to whether it is possible tosatisfy the reference error correction level within the referencelatency, correcting the error through a fixed error correction circuitor a variable error correction circuit, wherein the fixed errorcorrection circuit is configured to correct the first read data to anerror-free state, and wherein the variable error correction circuit isconfigured to correct the first read data to satisfy the reference errorcorrection level.
 27. The method of claim 26, wherein the providing ofthe first correction data to the host processor comprises furtherproviding a read response to the first read request together with thefirst correction data corrected by the variable error correctioncircuit, wherein the read response comprises an error flag set by thevariable error correction circuit.
 28. The method of claim 26, whereinthe fixed error correction circuit is configured to correct the firstread data in parallel with the variable error correction circuit togenerate second correction data and store the second correction data ina buffer.
 29. The method of claim 28, further comprising: receiving asecond read request for the first read data from the host processor; andproviding the second correction data stored in the buffer to the hostprocessor based on the CXL memory protocol.
 30. The method of claim 22,wherein the memory comprises at least one storage-class memory.
 31. Asystem comprising: a host processor comprising at least one coreconfigured to execute instructions; a Compute Express Link™ (CXL) deviceconnected to the host processor; and a memory configured to be accessedthrough the CXL device, wherein the CXL device is configured to receivea first read request from the host processor based on a CXL memoryprotocol, read first read data corresponding to the first read requestfrom the memory, and perform in parallel a first error correctionoperation and a second error correction operation, the first errorcorrection operation based on at least one of a reference latency or areference error correction level included in a first error correctionoperation, and the second error correction operation based on a maximumerror correction level on an error included in the first read data,wherein the first error correction operation is performed according towhether it is possible to satisfy the reference error correction levelwithin the reference latency.
 32. The system of claim 31, wherein theCXL device is configured to receive a device information request fromthe host processor based on a CXL input/output protocol, transmit deviceinformation to the host processor, and receive the first errorcorrection option from the host processor.
 33. The system of claim 32,wherein the device information is based on subordinate-to-master (S2M)transaction, wherein the device information comprises: a first fielddata corresponding to a logical device identifier, and a second fielddata corresponding to a load level, wherein the load level is determinedbased on at least one of a queued request, an internal resource usage ora bandwidth usage.
 34. The system of claim 32, wherein the first errorcorrection option is based on a master-to-subordinate (M2S) transaction.35. The system of claim 31, wherein first correction data generatedthrough the first error correction operation is provided to the hostprocessor based on the CXL memory protocol, and wherein secondcorrection data generated through the second error correction operationis stored in a buffer.
 36. The system of claim 35, wherein the CXLdevice is configured to generate a read response comprising an errorflag, and provide the read response together with the first correctiondata to the host processor.
 37. The system of claim 35, wherein the CXLdevice is configured to receive a second read request for the first readdata from the host processor, and provide the second correction datastored in the buffer to the host processor.
 38. The system of claim 37,wherein a time required to generate the first correction data throughthe first error correction operation is shorter than a time required togenerate the second correction data through the second error correctionoperation.
 39. A compute express link (CXL) device configured to controla memory accessed by the CXL device connected to a host processor, theCXL device comprising: a first interface circuit configured tocommunicate with the host processor based on CXL protocol; a secondinterface circuit configured to communicate with the memory; an errordetection circuit configured to detect an error present in first readdata read from the second interface circuit in response to a first readrequest received from the first interface circuit; a variable errorcorrection circuit configured to correct the detected error based on atleast one of a reference latency and a reference error correction levelincluded in a first error correction option; and a fixed errorcorrection circuit configured to correct the detected error in parallelwith an operation of the variable error correction circuit. provide thefirst read data to the variable error correction circuit so that thevariable error correction circuit corrects the error according towhether it is possible to satisfy the reference error correction levelwithin the reference latency.
 40. The CXL device of claim 39, whereinthe error detection circuit is configured to provide the first read datato the variable error correction circuit, wherein the variable errorcorrection circuit is configured to correct the error according towhether it is possible to satisfy the reference error correction levelwithin the reference latency.